Loss Function Research Articles

Application of improved DBN and GRU based on intelligent optimization algorithm in power load identification and prediction

Non intrusive load monitoring belongs to the key technologies of intelligent power management systems, playing a crucial role in smart grids. To achieve accurate identification and prediction of electricity load, intelligent optimization algorithms are introduced into deep learning optimization for improvement. A load recognition model combining sparrow search algorithm and deep confidence network is designed, as well as a gated recurrent network prediction model on the grounds of particle swarm optimization. The relevant results showed that the sparrow search algorithm used in the study performed well on the solution performance evaluation metrics with a minimum value of 0.209 for the inverse generation distance and a maximum value of 0.814 for the hyper-volume. The accuracy and recall values of the optimized load identification model designed in the study were relatively high. When the accuracy was 0.9, the recall rate could reach 0.94. The recognition accuracy of the model on the basis of the test set could reach up to 0.924. The lowest classification error was only 0.05. The maximum F1 value of the harmonic evaluation index of the bidirectional gated recurrent network optimized by particle swarm optimization converged to 90.06%. The loss function had been optimized by particle swarm optimization, and both the convergence value and convergence speed had been markedly enhanced. The average absolute error and root mean square error of the prediction model were both below 0.3. Compared to the bidirectional gated recurrent model before optimization, the particle swarm optimization strategy had a significant improvement effect on prediction details. In addition, the research method had superior recognition response speed and adaptability in real application environments. This study helps to understand the load demand of the power system, optimize the operation of the power grid, and strengthen the reliability, efficiency, and sustainability of the power system.

STAM-LSGRU: a spatiotemporal radar echo extrapolation algorithm with edge computing for short-term forecasting

With the advent of Mobile Edge Computing (MEC), shifting data processing from cloud centers to the network edge presents an advanced computational paradigm for addressing latency-sensitive applications. Specifically, in radar systems, the real-time processing and prediction of radar echo data pose significant challenges in dynamic and resource-constrained environments. MEC, by processing data near its source, not only significantly reduces communication latency and enhances bandwidth utilization but also diminishes the necessity of transmitting large volumes of data to the cloud, which is crucial for improving the timeliness and efficiency of radar data processing. To meet this demand, this paper proposes a model that integrates a spatiotemporal Attention Module (STAM) with a Long Short-Term Memory Gated Recurrent Unit (ST-ConvLSGRU) to enhance the accuracy of radar echo prediction while leveraging the advantages of MEC. STAM, by extending the spatiotemporal receptive field of the prediction units, effectively captures key inter-frame motion information, while optimizations to the convolutional structure and loss function further boost the model’s predictive performance. Experimental results demonstrate that our approach significantly improves the accuracy of short-term weather forecasting in a mobile edge computing environment, showcasing an efficient and practical solution for processing radar echo data under dynamic, resource-limited conditions.

Non-Differentiable Loss Function Optimization and Interaction Effect Discovery in Insurance Pricing Using the Genetic Algorithm

Non-Differentiable Loss Function Optimization and Interaction Effect Discovery in Insurance Pricing Using the Genetic Algorithm

Insulator defect detection in transmission line based on an improved lightweight YOLOv5s algorithm

As one of the most important components on the transmission line, the insulator is prone to failure, which affects the safe operation of electrical power system. Hence, it is crucial to accurately detect the insulator defects for further maintenance in time. Recently, with the development of artificial intelligence and target detection algorithms, the insulator defect detection has received more and more attention. However, there are still existing some difficulties: insufficient samples and low detection accuracy. To improve the accuracy of insulator defect detection, this paper proposes an auto-detection method based on an improved lightweight YOLOv5s model. First, this paper introduces the basic network frame of YOLOv5s and proposes an improved algorithm by utilizing the GIoU loss function, Mish activation function, and CBAM module. Then, performs data enhancement in the insulator dataset to enhance the robustness of the model. Finally, trains and tests the improved YOLOv5s model, and compares it with traditional target detection algorithms. Compared with traditional target detection algorithms, the AP value of the proposed algorithm in detecting insulator defects can be improved by 5 %. The results demonstrate the improved algorithm proposed in this paper can effectively identify and position the insulator defects.

Reducing Training Data Needs with Minimal Multilevel Machine Learning (M3L)

Abstract For many machine learning applications in science, data acquisition, not training, is the bottleneck even when avoiding experiments and relying on computation and simulation. 
Correspondingly, and in order to reduce cost and carbon footprint, training data efficiency is key. 
We introduce minimal multilevel machine learning (M3L) which 
optimizes training data set sizes using a loss function at multiple levels of reference data in order to minimize a combination of prediction error with overall training data acquisition costs (as measured by computational wall-times).
Numerical evidence has been obtained for calculated atomization energies and electron affinities of thousands of organic molecules at various levels of theory including HF, MP2, DLPNO-CCSD(T), DFHFCABS, PNOMP2F12, and PNOCCSD(T)F12, and treating them with 
basis sets TZ, cc-pVTZ, and AVTZ-F12.
Our M3L benchmarks for reaching chemical accuracy in distinct chemical compound sub-spaces 
indicate substantial computational cost reductions by factors of 
$\sim$ 1.01, 1.1, 3.8, 13.8, and 25.8 when compared to heuristic sub-optimal multilevel machine learning (M2L) for the data sets QM7b, QM9$^\mathrm{LCCSD(T)}$, EGP, QM9$^\mathrm{CCSD(T)}_\mathrm{AE}$, and QM9$^\mathrm{CCSD(T)}_\mathrm{EA}$, respectively.
Furthermore, we use M2L to investigate the performance for 76 density functionals when used within multilevel learning and building on the following levels drawn from the hierarchy of Jacobs Ladder:~LDA, GGA, mGGA, and hybrid functionals.
Within M2L and the molecules considered, 
mGGAs do not provide any noticeable advantage over GGAs. 
Among the functionals considered and in combination with LDA, the three on average top performing GGA and Hybrid levels for atomization energies on QM9 using M3L correspond respectively to PW91, KT2, B97D, and $\tau$-HCTH, B3LYP$\ast$(VWN5), and TPSSH.

LSTM-Autoencoder Deep Learning Model for Anomaly Detection in Electric Motor

Anomaly detection is the process of detecting unusual or unforeseen patterns or events in data. Many factors, such as malfunctioning hardware, malevolent activities, or modifications to the data’s underlying distribution, might cause anomalies. One of the key factors in anomaly detection is balancing the trade-off between sensitivity and specificity. Balancing these trade-offs requires careful tuning of the anomaly detection algorithm and consideration of the specific domain and application. Deep learning techniques’ applications, such as LSTMs (long short-term memory algorithms), which are autoencoders for detecting an anomaly, have garnered increasing attention in recent years. The main goal of this work was to develop an anomaly detection solution for an electrical machine using an LSTM-autoencoder deep learning model. The work focused on detecting anomalies in an electrical motor’s variation vibrations in three axes: axial (X), radial (Y), and tangential (Z), which are indicative of potential faults or failures. The presented model is a combination of the two architectures; LSTM layers were added to the autoencoder in order to leverage the LSTM capacity for handling large amounts of temporal data. To prove the LSTM efficiency, we will create a regular autoencoder model using the Python programming language and the TensorFlow machine learning framework, and compare its performance with our main LSTM-based autoencoder model. The two models will be trained on the same database, and evaluated on three primary points: training time, loss function, and MSE anomalies. Based on the obtained results, it is clear that the LSTM-autoencoder shows significantly smaller loss values and MSE anomalies compared to the regular autoencoder. On the other hand, the regular autoencoder performs better than the LSTM, comparing the training time. It appears then, that the LSTM-autoencoder presents a superior performance although it was slower than the standard autoencoder due to the complexity of the added LSTM layers.

Self-supervised monocular visual odometry based on cross-correlation

Visual odometry constitutes a critical component in enabling autonomous navigation. However, the existing methods are limited by the feature extraction and matching accuracy, and cannot show good real-time performance while combining accuracy and robustness. In this paper, we propose a novel monocular visual odometry framework based on cross-correlation. The framework starts with a parameter-sharing Siamese network to build feature extractors that can simultaneously process multiple images as inputs. Moreover, we design cross-correlation modules and define a cross-correlation matrix to describe the strength of correlation between different parts of the input feature maps, reflecting the rotational and translational transformations of the input images. Furthermore, a novel loss function is introduced to impose constraints on the network. Additionally, a fully convolutional network is designed for pose estimation, computing poses alterations from the structure of the cross-correlation matrix. Channel attention and spatial attention mechanisms are introduced to improve the performance. More importantly, our method innovatively uses time intervals as labels, enables self-supervised training, and relies only on a monocular camera. Experimental results on the KITTI visual odometry dataset and the Oxford Robotcar Dataset show that our method produces competitive performance, demonstrating the superiority of the proposed method.

Semi-supervised method for visual detection of automotive paint defects

Automotive paint defect detection plays a crucial role in the automotive production process. Current research on visual defect detection methods is mainly based on supervised learning, which requires a large number of labeled image samples for model training. The labeling work is not only time consuming but also expensive, seriously hindering the testing and application of these models in practice. To address this issue, this study proposes a new method for automotive paint defect detection based on a semi-supervised training strategy. First, a semi-supervised automotive paint defect detection framework, which can use labeled and unlabeled samples to reduce the cost of data labeling effectively, is presented. Then, a spatial pyramid pooling fast external attention module that introduces an external attention mechanism is proposed to improve the traditional YOLOv7 network structure, called YOLOv7-EA, to obtain good detection performance. This network acts as a detector to generate high-quality pseudo labels for the unlabeled samples, providing additional data to train the model; meanwhile, it performs the final detection task. Lastly, a Wise-intersection over union loss function that considers the quality of the anchor box is introduced to reduce the interference of low-quality samples and improve the convergence speed and detection accuracy of the model. Using this method, we can accomplish the task of automotive paint defect detection with a small number of labeled image samples. Experimental results on the automotive paint defect dataset show that mean average precision (mAp)@.5, mAp@.75, and mAp@.5:.95 are superior to other methods under the condition of 10% and 15% labeled data, achieving good defect detection performance.

Improved YOLOv8 and SAHI Model for the Collaborative Detection of Small Targets at the Micro Scale: A Case Study of Pest Detection in Tea

Pest target identification in agricultural production environments is challenging due to the dense distribution, small size, and high density of pests. Additionally, changeable environmental lighting and complex backgrounds further complicate the detection process. This study focuses on enhancing the recognition performance of tea pests by introducing a lightweight pest image recognition model based on the improved YOLOv8 architecture. First, slicing-aided fine-tuning and slicing-aided hyper inference (SAHI) are proposed to partition input images for enhanced model performance on low-resolution images and small-target detection. Then, based on an ELAN, a generalized efficient layer aggregation network (GELAN) is designed to replace the C2f module in the backbone network, enhance its feature extraction ability, and construct a lightweight model. Additionally, the MS structure is integrated into the neck network of YOLOv8 for feature fusion, enhancing the extraction of fine-grained and coarse-grained semantic information. Furthermore, the BiFormer attention mechanism, based on the Transformer architecture, is introduced to amplify target characteristics of tea pests. Finally, the inner-MPDIoU, based on auxiliary borders, is utilized as a replacement for the original loss function to enhance its learning capacity for complex pest samples. Our experimental results demonstrate that the enhanced YOLOv8 model achieves a precision of 96.32% and a recall of 97.95%, surpassing those of the original YOLOv8 model. Moreover, it attains an mAP@50 score of 98.17%. Compared to Faster R-CNN, SSD, YOLOv5, YOLOv7, and YOLOv8, its average accuracy is 17.04, 11.23, 5.78, 3.75, and 2.71 percentage points higher, respectively. The overall performance of YOLOv8 outperforms that of current mainstream detection models, with a detection speed of 95 FPS. This model effectively balances lightweight design with high accuracy and speed in detecting small targets such as tea pests. It can serve as a valuable reference for the identification and classification of various insect pests in tea gardens within complex production environments, effectively addressing practical application needs and offering guidance for the future monitoring and scientific control of tea insect pests.

G2NPAN: GAN-guided nuance perceptual attention network for multimodal medical fusion image quality assessment

Multimodal medical fusion images (MMFI) are formed by fusing medical images of two or more modalities with the aim of displaying as much valuable information as possible in a single image. However, due to the different strategies of various fusion algorithms, the quality of the generated fused images is uneven. Thus, an effective blind image quality assessment (BIQA) method is urgently required. The challenge of MMFI quality assessment is to enable the network to perceive the nuances between fused images of different qualities, and the key point for the success of BIQA is the availability of valid reference information. To this end, this work proposes a generative adversarial network (GAN) -guided nuance perceptual attention network (G2NPAN) to implement BIQA for MMFI. Specifically, we achieve the blind evaluation style via the design of a GAN and develop a Unique Feature Warehouse module to learn the effective features of fused images from the pixel level. The redesigned loss function guides the network to perceive the image quality. In the end, the class activation mapping supervised quality assessment network is employed to obtain the MMFI quality score. Extensive experiments and validation have been conducted in a database of medical fusion images, and the proposed method is superior to the state-of-the-art BIQA method.

Uncovering the Potential of Industrial Waste: Turning Discarded Resources into Sustainable Advanced Materials

Abstract To identify solutions for utilizing industrial wastes (slag, sludge, fly ash, and bottom ash) in the development of pro-environmental systems, this paper reviews existing literature and explores material characterization methods. A review analysis was conducted to uncover their potential, and it was found that many utilizations can be made to recycle them into sustainable materials. Structure characterization, optical-dielectric function, and morphology analysis were analyzed to confirm the recent review. Structural properties are represented by crystallite size and crystallinity, where nickel slag shows the smallest distribution (55±10) nm with the highest crystallinity (25.42%), indicating its association with uniformity of response to external disturbances and micro-structural stability. From the analysis of optical and dielectric functions via Kramers Kronig relation, the widening of ∆(LO-TO) was shown by fly ash and slag indicating stable bonding suitable as x-ray shielding and sensor, while high electron loss functions were shown by sludge and bottom ash indicating high potential electronic transition performance. Based on their chemical and physical properties, these industrial wastes are not limited as construction materials and can even be upgraded to more advanced development materials.

WaveletDFDS-Net: A Dual Forward Denoising Stream Network for Low-Dose CT Noise Reduction

The challenge of denoising low-dose computed tomography (CT) has garnered significant research interest due to the detrimental impact of noise on CT image quality, impeding diagnostic accuracy and image-guided therapies. This paper introduces an innovative approach termed the Wavelet Domain Dual Forward Denoising Stream Network (WaveletDFDS-Net) to address this challenge. This method ingeniously combines convolutional neural networks and Transformers to leverage their complementary capabilities in feature extraction. Additionally, it employs a wavelet transform for efficient image downsampling, thereby preserving critical information while reducing computational requirements. Moreover, we have formulated a distinctive dual-domain compound loss function that significantly enhances the restoration of intricate details. The performance of WaveletDFDS-Net is assessed by comparative experiments conducted on public CT datasets, and results demonstrate its enhanced denoising effect with an SSIM of 0.9269, PSNR of 38.1343 and RMSE of 0.0130, superior to existing methods.

High‐dimensional differential networks with sparsity and reduced‐rank

AbstractDifferential network analysis plays a crucial role in capturing nuanced changes in conditional correlations between two samples. Under the high‐dimensional setting, the differential network, that is, the difference between the two precision matrices are usually stylized with sparse signals and some low‐rank latent factors. Recognizing the distinctions inherent in the precision matrices of such networks, we introduce a novel approach, termed ‘SR‐Network’ for the estimation of sparse and reduced‐rank differential networks. This method directly assesses the differential network by formulating a convex empirical loss function with ‐norm and nuclear norm penalties. The study establishes finite‐sample error bounds for parameter estimation and highlights the superior performance of the proposed method through extensive simulations and real data studies. This research significantly contributes to the advancement of methodologies for accurate analysis of differential networks, particularly in the context of structures characterized by sparsity and low‐rank features.

Comparison of some Bayesian estimation methods for type-I generalized extreme value distribution with simulation

The Weibull distribution is considered one of the Type-I Generalized Extreme Value (GEV) distribution, and it plays a crucial role in modeling extreme events in various fields, such as hydrology, finance, and environmental sciences. Bayesian methods play a strong, decisive role in estimating the parameters of the GEV distribution due to their ability to incorporate prior knowledge and handle small sample sizes effectively. In this research, we compare several shrinkage Bayesian estimation methods based on the squared error and the linear exponential loss functions. They were adopted and compared by the Monte Carlo simulation method. The performance of these methods is assessed based on their accuracy and computational efficiency in estimating the scale parameter of the Weibull distribution. To evaluate their performance, we generate simulated datasets with different sample sizes and varying parameter values. A technique for pre-estimation shrinkage is suggested to enhance the precision of estimation. Simulation experiments proved that the Bayesian shrinkage estimator and shrinkage pre-estimation under the squared loss function method are better than the other methods because they give the least mean square error. Overall, our findings highlight the advantages of shrinkage Bayesian estimation methods for the proposed distribution. Researchers and practitioners in fields reliant on extreme value analysis can benefit from these findings when selecting appropriate Bayesian estimation techniques for modeling extreme events accurately and efficiently.

FAST: Filter-Adapted Spatio-Temporal Sampling for Real-Time Rendering

Stochastic sampling techniques are ubiquitous in real-time rendering, where performance constraints force the use of low sample counts, leading to noisy intermediate results. To remove this noise, the post-processing step of temporal and spatial denoising is an integral part of the real-time graphics pipeline. The main insight presented in this paper is that we can optimize the samples used in stochastic sampling such that the post-processing error is minimized. The core of our method is an analytical loss function which measures post-filtering error for a class of integrands --- multidimensional Heaviside functions. These integrands are an approximation of the discontinuous functions commonly found in rendering. Our analysis applies to arbitrary spatial and spatiotemporal filters, scalar and vector sample values, and uniform and non-uniform probability distributions. We show that the spectrum of Monte Carlo noise resulting from our sampling method is adapted to the shape of the filter, resulting in less noisy final images. We demonstrate improvements over state-of-the-art sampling methods in three representative rendering tasks: ambient occlusion, volumetric ray-marching, and color image dithering. Common use noise textures, and noise generation code is available at https://github.com/electronicarts/fastnoise.

Remote Measurement of Tide and Surge Using a Deep Learning System with Surveillance Camera Images

The latest progress in deep learning approaches has garnered significant attention across a variety of research fields. These techniques have revolutionized the way marine parameters are measured, enabling automated and remote data collection. This work centers on employing a deep learning model for the automated evaluation of tide and surge, aiming to deliver accurate results through the analysis of surveillance camera images. A mode of deep learning based on the Inception v3 structure was applied to predict tide and storm surges from surveillance cameras located in two different coastal areas of Italy. This approach is particularly advantageous in situations where traditional tide sensors are inaccessible or distant from the measurement point, especially during extreme events that require accurate surge measurements. The conducted experiments illustrate that the algorithm efficiently measures tide and surge remotely, achieving an accuracy surpassing 90% and maintaining a loss value below 1, evaluated through Categorical Cross-Entropy Loss functions. The findings highlight its potential to bridge the gap in data collection in challenging coastal environments, providing valuable insights for coastal management and hazard assessments. This research contributes to the emerging field of remote sensing and machine learning applications in environmental monitoring, paving the way for enhanced understanding and decision-making in coastal regions.

SEMScene: Semantic-Consistency Enhanced Multi-Level Scene Graph Matching for Image-Text Retrieval

Image-text retrieval, a fundamental cross-modal task, performs similarity reasoning for images and texts. The primary challenge for image-text retrieval is cross-modal semantic heterogeneity, where the semantic features of visual and textual modalities are rich but distinct. Scene graph is an effective representation for images and texts as it explicitly models objects and their relations. Existing scene graph based methods have not fully taken the features regarding various granularities implicit in scene graph into consideration (e.g. triplets), the inadequate feature matching incurs the absence of non-trivial semantic information (e.g. inner relations among triplets). Therefore, we propose a S emantic-Consistency E nhanced M ulti-Level Scene Graph Matching (SEMScene) network, which exploits the semantic relevance between visual and textual scene graphs from fine-grained to coarse-grained. Firstly, under the scene graph representation, we perform feature matching including low-level node matching, mid-level semantic triplet matching, and high-level holistic scene graph matching. Secondly, to enhance the semantic-consistency for object-fused triplets carrying key correlation information, we propose a dual-step constraint mechanism in mid-level matching. Thirdly, to guide the model to learn the semantic-consistency of matched image-text pairs, we devise effective loss functions for each stage of the dual-step constraint. Comprehensive experiments on Flickr30K and MS-COCO datasets demonstrate that SEMScene achieves state-of-the-art performances with significant improvements.

FS_YOLOv8: A Deep Learning Network for Ground Fissures Instance Segmentation in UAV Images of the Coal Mining Area

Abstract. The ground fissures caused by coal mining have seriously affected the ecological environment of the land. Timely and accurate identification and landfill treatment of ground fissures can avoid secondary geological disasters in coal mine areas. At present, the fissure identification methods based on deep learning show excellent performance on roads and walls, etc. Nevertheless, the automatic and reliable segmentation of ground fissures in remote sensing images poses a challenge for deep learning networks, due to the diverse and complex texture information included in the mining ground fissures and background. To overcome these challenges, we propose an improved YOLOv8 instance segmentation network to automatically and efficiently segment the ground fissures in coal mining areas. In detail, a model called FS_YOLOv8 is proposed. The DSPP (Dynamic Snake convolutional Pyramid Pooling) module is incorporated into the FS_YOLOv8 model to establish a multi-scale dynamic snake convolution feature aggregation structure. This module replaces the conventional convolution found in the SPPF module of YOLOv8 and aims to enhance the model's ability to extract features related to fissures with tubular structures. Furthermore, the D-LKA (Deformable Large Kernel Attention) module is employed to autonomously collect fissure context information. To enhance the detection capability of challenging samples in remote sensing images with intricate background and fissure texture, we employ a Slide Loss function. Ultimately, the ground fissure dataset of unmanned aerial vehicle (UAV) images in coal mine areas is subjected to experimental analysis. The experimental findings demonstrate that FS_YOLOv8 exhibits exceptional proficiency in segmenting ground fissures within intricate and expansive mining areas.

Soybean seedling detection and counting from UAV images based on an improved YOLOv8 Network

Abstract. The utilization of unmanned aerial vehicle (UAV) for soybean seedling detection is an effective way to estimate soybean yield, which plays a crucial role in agricultural planning and decision-making. However, the soybean seedlings objects in the UAV image are small, in clusters, and occluded each other, which makes it very challenging to achieve accurate object detection and counting. To address these issues, we optimize the YOLOv8 model and propose a GAS-YOLOv8 network, aiming to enhance the detection accuracy for the task of soybean seedling detection based on UAV images. Firstly, a global attention mechanism (GAM) is incorporated into the neck module of YOLOv8, which reallocates weights and prioritizes global information to more effectively extract soybean seedling features. Secondly, the CIOU loss function is replaced with the SIOU loss, which includes an angle loss term to guide the regression of bounding boxes. Experimental results show that, on the soybean seedling dataset, the proposed GAS-YOLOv8 model achieves a 1.3% improvement in mAP@0.5 and a 6% enhancement in detection performance in dense seedling areas, when compared to the baseline model YOLOv8s.When compared to other object detection models (YOLOv5, Faster R-CNN, etc.), the GAS-YOLOv8 model similarly achieved the best detection performance. These results demonstrate the effectiveness of the GAS-YOLOv8 in detecting dense soybean seedlings, providing more accurate theoretical support for subsequent yield estimation.

Electronic heat conductivity in a two-temperature state

Heat transport in solids is governed by two fundamental contributions, atomic and electronic. The electronic energy transport in transient excited states is a defining factor in the problem of ultrafast material irradiation. Here, we calculate the electronic heat conductivity at elevated electron temperatures up to 40,000 K. We apply the novel combined method of tight binding formalism to calculate the electron-phonon contribution to the electronic heat conductivity, and the linear response theory (in the single-pole Ritchie-Howie loss function approximation) for its electron-electron counterpart, implemented in the hybrid code XTANT-3. It allows us to evaluate the electronic heat conductivity in a wide range of materials – fcc metals: Al, Ca, Ni, Cu, Sr, Y, Zr, Rh, Pd, Ag, Ir, Pt, Au, and Pb; hcp metals: Mg, Sc, Ti, Co, Zn, Tc, Ru, Cd, Hf, Re, and Os; bcc metals: V, Cr, Fe, Nb, Mo, Ba, Ta, and W; other metals: Sn, Ga, In, Mn, Te, and Se; semimetal graphite; semiconductors – group IV: Si, Ge, and SiC; group III-V: AlAs, AlP, GaP, GaAs, and GaSb; oxides: ZnO, TiO2, and Cu2O; and others: PbI2, ZnS, and B4C.